Cumulative Innovation & Open Disclosure of Intermediate Results: Evidence from a Policy Experiment in Bioinformatics

Recent calls for greater openness in our private and public innovation systems have particularly urged for more open disclosure and granting of access to intermediate works–early results, algorithms, materials, data and techniques–with the goals of enhancing overall research and development productivity and enhancing cumulative innovation. To make progress towards understanding implications of such policy changes we devised a large-scale field experiment in which 733 subjects were divided into matched independent subgroups to address a bioinformatics problem under either a regime of open disclosure of intermediate results or, alternatively, one of closed secrecy around intermediate solutions. We observe the cumulative innovation process in each regime with fine-grained measures and are able to derive inferences with a series of cross-sectional comparisons. Open disclosures led to lower participation and lower effort but nonetheless led to higher average problem-solving performance by concentrating these lesser efforts on the most performant technical approaches. Closed secrecy produced higher participation and higher effort, while producing less correlated choices of technical approaches that participants pursued, resulting in greater individual and collective experimentation and greater dispersion of performance. We discuss the implications of such changes to the ongoing theory, evidence and policy considerations with regards to cumulative innovation. (JEL O3, JO, D02) * Boudreau: London Business School and Institute of Quantitative Social Science at Harvard University, Regent’s Park, London, U.K. NW1 4SA, fax: +44 (0)20 7000 8701, telephone: +44 (0)20 7000 8455, e-mail: [email protected]; Lakhani: Harvard Business School and Institute of Quantitative Social Science at Harvard University, email: [email protected]. We are grateful to members of the Harvard Medical School communities for their contribution of considerable attention and resources to this project, including Ramy Arnaout, Eva Guinan and Lee Nadler. We also thank managers at TopCoder, including Jack Hughes, Rob Hughes, Mike Lydon, and Ira Heffan, who provided invaluable assistance in carrying out all aspects of the experiment and in designing and implementing the experimental platform. We thank expert computation and data scientists Po-Ru Loh, Hernán Amiune and Xiaoshi Lu for careful technical evaluations of the data algorithms developed within the experiment. We would like to thank several people for their comments, including: Lee Branstetter, Wesley Cohen, Carliss Baldwin, Erik Brynjolfsson, Chaim Ferschtman, Rebecca Henderson, Nicola Lacetera, Alan McCormack, Petra Moser, Ramana Nanda, Richard Nelson, Catherine Tucker and seminar participants at London Business School, Tel Aviv University, the National Bureau of Economic Research (NBER), and the Roundtable for Engineering Entrepreneurship Research (REER). Onal Vural provided excellent research assistance. All errors are our own. Boudreau would like to acknowledge financial support from a London Business School Research and Materials Development Grant and the University of Toronto, Rotman School of Management. Lakhani would like to acknowledge the financial support of the HBS Division of Research and Faculty Development. A Google Faculty Research Grant supported both authors.